1. Field of the Invention
The present invention relates to techniques for measuring a status change entailing motions and/or actions of a human being, an animal or a machine (generically called the object under observation hereunder). More particularly, the invention relates to a method for recognizing such motions and/or actions, as well as to an apparatus adopting that method and a system comprising that apparatus.
2. Description of Prior Art
The basic prior art pertaining to measuring human motions and work is described illustratively in xe2x80x9cErgonomics Illustratedxe2x80x9d (by Kageo Noro, a Japanese publication from Nippon Kikaku Kyokai, Feb. 14, 1990, pp. 538-544). This publication discloses typical methods for measuring motions and work of humans through VTR-based observation or by direct visual observation.
Japanese Patent Publication No. Hei 7-96014 discloses a technique for using acceleration sensors to measure motions and the work of humans. This technique involves acquiring vibration waveforms from acceleration sensors attached to the human body as the latter performs each motion, subjecting the acquired vibration waveforms to analog-digital conversion, composing discrete A/D converted values thus obtained into a vibration pattern table, and comparing for judgment any A/D converted value resulting from an input vibration waveform with the vibration pattern table in synchronism with a timing signal output by a clock in appropriate timings. This technique permits recognition processing in the same motion or at the same speed as that in or at which the vibration pattern table is created.
The initially-mentioned conventional methods for measuring motions and work of humans through VTR-based or direct visual observation have the following major disadvantages:
(1) Analysis through VTR-based or direct visual observation requires the observer to continuously record positions of the object under observation and the work done thereby. It takes many hours and enormous effort for the observer to perform the task.
(2) Blind spots of the object under observation cannot be inspected through VTR-based or direct visual observation.
(3) If the object under observation is in motion, the observer must follow the object by moving likewise.
(4) The object under observation is liable to remain conscious of the VTR or of the eyes of the observer.
(5) In observing where the articulations of the object under observation are positioned, the conventional methods merely measure and reproduce the articulation positions; the methods do not allow actions or work of the object to be recognized automatically. It takes human observers to recognize the reproduced motions.
The conventional technique subsequently mentioned above has the following major disadvantages:
(6) Pitch differences are apt to occur between motions of walking and running. For example, a leisurely walk and a brisk walk fall within the same xe2x80x9cwalkingxe2x80x9d category but have different speed""s entailing different lengths in the time base direction between the vibration pattern table and input vibration wave forms. This makes it impossible to detect correlations between the vibration waveform derived from the actual walk and the putatively corresponding vibration pattern table. The result is that motions of the same kind are erroneously recognized as two different motions. To recognize correctly the same kind of motions having different pitches requires additionally furnishing vibration pattern tables corresponding to different pitches. Furthermore, because pitches usually vary continuously, it takes many more vibration patterns to achieve more precise recognition. In short, to accomplish recognition of ever-higher precision requires a progressively large number of vibration pattern tables.
(7) Where a plurality of motions are to be recognized in combination, such as when the object is fanning himself while walking or when the object is walking inside a train in motion, it is necessary to establish a distinct vibration pattern for each of the combinations of motions (e.g., xe2x80x9cfanning during walking,xe2x80x9d xe2x80x9cwalking inside a running train,xe2x80x9d etc.) In addition, where the object is, for example fanning himself while walking, it is also necessary to establish a vibration pattern for a case in which the fan is facing upward the moment the subject""s foot contacts the ground, and another vibration pattern for a case where the fan is facing downward the moment the subject""s foot touches the ground. In practice, viable recognition requires setting up a very large number of vibration pattern tables to address diverse instances of motions.
(8) The conventional technique measures only the acceleration applied to the human body and recognizes that parameter as data subject to recognition. This makes it difficult to recognize motions such as twisting or like motion of the body characterized by angular acceleration.
(9) The objective of recognition is limited to intermittent motions, whereas a human action is generally achieved as a combination of a plurality of motions. For example, the action of xe2x80x9csitting on a chairxe2x80x9d is composed of motions xe2x80x9cwalking,xe2x80x9d xe2x80x9cstoppingxe2x80x9d and xe2x80x9csittingxe2x80x9d executed continuously in that order. The disclosed conventional technique has yet to address ways to recognize correctly the action made up of a plurality of motions.
An object of the present invention is to overcome the above and other deficiencies and disadvantages of the prior art and to provide a method for recognizing with more precision motions and actions of a moving object such as humans, animals and machines, as well as an apparatus adopting that method and a system comprising that apparatus.
A more specific object of the present invention is to provide a method, an apparatus and a system for recognition capable of achieving the following technical objectives:
(1) Motions, actions and work are automatically recognized so that the observer is free of undue operative burdens.
(2) Blind spots such as shadows of objects are eliminated.
(3) Observation will not be hampered or interrupted when the object under observation is in motion.
(4) The object under observation is free of burdens associated with conventional observation schemes.
(5) The object under observation is not merely measured for its motions, but also subjected to the recognition or estimation of its motions, actions and work on the basis of measured results.
(6) Measurements are not influenced by speeds of motions.
(7) A plurality of motions are recognized in combination.
(8) More kinds of motion than ever are recognized with reference to parameters other than acceleration.
(9) The object under observation (biological or mechanical) is not merely measured for its intermittent motions, but also subjected to the recognition or estimation of its motions, actions and work on the basis of a history of motions.
In carrying out the invention and according to one aspect thereof, there is provided a recognition apparatus for recognizing motions and actions of an object under observation, the recognition apparatus comprising: measurement means attached to the object under observation in order to measure a status change entailing the motions and actions of the object under observation; characteristic quantity extraction means for extracting characteristic quantities from the measurements taken by the measurement means; storage means for storing characteristic quantities of the motions and actions to be recognized by the recognition apparatus; recognition means for recognizing the motions and actions of the object under observation in accordance with the characteristic quantities extracted from the measurements and with the stored characteristic quantities; and output means for outputting a recognized result.
More specifically, a method, an apparatus and a system for recognition according to the invention illustratively comprise steps or means for performing the following:
(1) measuring instruments are attached to the object under observation. The instruments automatically take measurements of a status change entailing motions and actions of the object under observation. The measured data are transmitted to the observer.
(2) The status change of the object under observation is measured by use of an acceleration sensor, a velocity sensor or a position sensor free of visual or audio means.
(3) Observed results are transmitted by radio.
(4) Status changes are measured at a small number of observation points where these changes entailing motions and actions of the object under observation are pronounced.
(5) Characteristic quantities of typical motions or actions are extracted in advance. The predetermined characteristic quantities are compared in terms of correlation with characteristic quantities extracted from the measured data. From the comparison, a motion or an action represented by the characteristic quantities of high correlation is output as a recognized result.
(6) A previously measured waveform representing a single motion is normalized for duration from the time the motion is started until it ends. Normalization involves regarding the duration from start to end of a motion as xe2x80x9c1.xe2x80x9dIllustratively, two steps of a walk constitute one cycle when normalized. Thereafter, characteristic components of each motion are extracted through functionalization, Fourier transformation or wavelet transformation. Coefficients used in the normalization or the extracted components from the transformation (Fourier, etc.) are established as characteristic quantities of the motions to be referenced. When the start and end of a motion are to be recognized from an input waveform derived from the motion, the waveform is normalized on the basis of the motion duration. Thereafter, coefficients or characteristic components are extracted through the above-mentioned functionalization or Fourier/wavelet transformation. The extracted coefficients or components are compared with the previously stored characteristic quantities for motion recognition.
(7) The characteristic quantities of different motions are superposed on each other. The result of the overlaying is recognized through observation. For example, the characteristic quantities of xe2x80x9cwalkingxe2x80x9d and those of xe2x80x9cfanning oneselfxe2x80x9d are superposed on each other.
(8) For observation of motions and actions, not only acceleration data bus also other physical quantity data such as the velocity, position, angular acceleration, angular velocity, rotation angle and biological data entailing the motions and actions of the object under observation are acquired. The data thus obtained are used in the recognition process.
(9) Actions associated with a history of motions are previously established as associated motions. A history of observed and recognized motions is compared with the associated motions, whereby the actions of the object under observation are associated.
The steps or means outlined above accomplish illustratively the following effects when carried out or implemented:
(1) Measured data are automatically sent in, freeing the observer of operative chores conventionally associated with data gathering.
(2) The sensors used leave no blind spots.
(3) Because measurements are transmitted by radio, the object under observation may move about freely without the observer having to track him or her.
(4) A smaller number of observation points than in conventional setups reduce observation-related burdens on the object under observation. Without the observer watching the object, the latter is free from psychological constraints associated with the observer""s glances.
(5) The automatic recognition process makes it unnecessary for the observer directly to handle raw data measured.
(6) The normalized time base permits motion recognition free of the influence by motion velocity.
(7) Because characteristic quantities are considered in combination, more complicated motions can also be recognized.
(8) Recognition is carried out through the use of not only acceleration but also other physical quantities. This makes it possible to recognize motions characterized only insignificantly by acceleration.
(9) An action made up of a plurality of motions (called work hereunder) is also recognized.